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AUTOMATICALLY DETECTING OBJECTS 
BACKGROUND 

[0001] The present invention pertains to the technical field of object detection, and in 

particular to techniques for detecting any moving object in front of a vehicle. 

[0O02] Japanese Kokai Patent Application No. Hei 10[1998]-2 14326 discloses an 

automatic guidance system that uses Hough transformation to detect road edges from images 
in front of a vehicle. A string of detected edges is arranged side-by-side on a straight line and 
is subjected to Hough transformation to detect the road edges. For the Hough transformation, 
as the points that form the road edges are detected, curves are drawn in the parameter space, 
and the accumulated values of the points on the curves have to be added. This processing 
may be performed for all of the points that form the road edges, in which case the processing 
load is heavy. It would be desirable to provide an less computationally-intensive technique to 
detect road edges or other boundary lines in the path of a vehicle. 

SUMMARY 

[0003] In accordance with one aspect of the invention, an apparatus is provided for 

detecting objects in one or more images captured by an image pickup device mounted on a 
vehicle. The apparatus includes a memory on which is stored pixels of at least one image 
captured by the image pickup device; and a controller operatively coupled to the memory. 
The controller is adapted to compute velocity information for each pixel in the image; 
extract those pixels having a velocity component based on the velocity information; detect 
oblique lines composed of those extracted pixels having a velocity component, and generate a 
signal indicative of the road edge in the image based on the oblique lines. 

[0004] In accordance with another aspect of the invention, a method is provided for 

detecting objects in an image captured from in front of a vehicle. The method includes 
processing the image to compute velocity information for each pixel in the image; extracting 
pixels having a velocity component on the basis of the velocity information of the pixels of 
the images computed by the velocity information computing means; detecting oblique lines 
made of pixels having a velocity component and extracted by the pixel extracting means; and 
detecting the boundary line on the road present in the image in front of the vehicle on the 
basis of the oblique line detected by the oblique line detecting means. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



(0005] The description herein makes reference to the accompanying drawings 

wherein like reference numerals refer to like parts throughout the several views, and wherein: 
[0006] Figure 1 is a block diagram of an object detection device in accordance with a 

first embodiment of the invention; 

[0O07] Figure 2A is a side elevation of a vehicle in which the device of Figure 1 has 

been installed, including a camera; 

[0008] Figure 2B is a top plan view of the vehicle shown in Figure 2A; 

[0009] Figure 3 is a diagram of an image captured by the camera shown in Figure 2A; 

[0010] Figure 4A shows an image in which an edge in a stored image has been 

extracted; 

[0011] Figure 4B shows the image of Figure 4A after it has been subjected to a line- 

thinning processes; 

[0012] Figure 4C shows the image of Figure 43 after it has been expanded to a fixed 

width; 

[0013] Figure 5 is a diagram of an image captured by the camera shown in Figure 2A, 

including a depiction of a velocity image; 

[0014] Figure 6 is a diagram of an image captured by the camera shown in Figure 2A, 

including a road edge detecting region that is established in the velocity image; 
[0015] Figure 7 is a diagram of the image shown in Figure 6, illustrating the 

extraction of the pixels having a velocity component towards the center of the 
two-dimensional road model; 

[0016] Figure 8 is a diagram of the image shown in Figure 6, illustrating the detection 

of an object field containing a pedestrian moving towards center of the two-dimensional road 
model, by grouping the pixels having the same velocity component towards center in the 
two-dimensional road model in the velocity image; 

[0017] Figure 9 is a diagram of the image shown in Figure 6, illustrating the direction 

of movement of the oblique lines in the velocity image when pitching takes place; 
[0018] Figure 10 is a diagram of the image shown in Figure 6, illustrating the 

transformation of a two-dimensional road model into a three-dimensional road model; 
[0019] Figure 1 1 is a diagram of the relative positional relationships among the 

vehicle shown in Figure 2A, the road edges and the object detected in real space; 



[0020] Figure 1 2A is a flow chart of the operation of the object detection device 

shown in Figure I ; 

[0021] Figure 12B is a continuation of the flow chart shown in Figure 12A; 

[0022] Figure 1 3 is a diagram of an image captured by the camera shown in Figure 

2 A, in accordance with a second embodiment of the invention; 
[00231 Figure 14 is a diagram of the image shown in Figure 13, illustrating a 

depiction of the velocity component; 

[0024] Figure 15 is a diagram of the image shown in Figure 13, illustrating the 

formation of regions for detecting boundary lines in the velocity image; 

[0025] Figure 16A is a diagram of the image shown in Figure 13, illustrating the 

detection of candidates for oblique lines and the application of a spatial filter; 

[0026] Figure 16B is a chart showing the identification of an oblique line candidate ii 

accordance with the second embodiment of the invention; 

[0027] Figure 17 is a diagram of the image shown in Figure 13, illustrating the 

detection of boundary lines on the road; 

[0028] Figure 1 8 A is the flow chart illustrating the processing of object detecting 

device in accordance with the second embodiment of the invention; 

[0029] Figure 1 8B is a continuation of the flow chart of Figure 1 8 A; 

[0030] Figure 19 is a block diagram showing a detailed structure an edge-width 

standardizing section; 

[0031] Figure 20 is a view illustrating thin-line condensing operation and expanding 

operation by the edge-width standardizing section shown in Figure 19. 

[0032] Figure 21 is a view illustrating a count up operation of a balloting module. 



DETAILED DESCRIPTION 

First embodiment 

[0033] Figure 1 is a block diagram illustrating an object detection device in 

accordance with a first embodiment of the present invention. Object detecting device 100 is 
mounted on the vehicle. It has the following parts: camera 101, image memory 102 that 
stores the images taken with the camera 101, controller 103 that executes image processing of 
the images taken with camera 101 and stored in image memory 102, counter memory 104 
that stores the count value of the pixel counter to be explained later, speaker 1 05 that outputs 



sounds, automatic braking device 106 that controls the vehicle brakes, and vehicle speed 
sensor 1 07 that detects the speed of the vehicle. 

[0034] Camera 101 is a high-speed camera having CCD, CMOS or other image 

pickup elements, and it can take consecutive images of objects from ahead of the vehicle. The 
image frames are output to image memory 102. The camera 101 is arranged on the upper 
front portion of the vehicle, as shown in Figure 2. Its sight of view axial direction Z is in front 
of the vehicle. It is set such that horizontal axis X of the imaging area is parallel to the ground 
surface, and vertical axis Y of the imaging area is perpendicular to the ground surface. The 
images taken by camera 1 01 are consecutively output and stored in image memory 102. 

[0035] Figure 3 is a diagram illustrating an example of the image taken with camera 

101. Image 3a is taken with camera 101 in an xy coordinate system (camera coordinates) with 
the origin at the upper-left portion of the image, and with the X-axis defined from left to right 
and Y-axis defined from upper to lower. In Figure 3, image 3a includes road edges A and B 
formed as edge bumps, white lines or guard rails or the like positioned on the left/right sides 
of the road, and pedestrian C present in front of the vehicle. 

[0036] Controller 103 processes image 3a stored in image memory 102, as will be 

explained later, detects the object present ahead of the vehicle itself, and when there is a 
danger of collision of the vehicle itself with the object present ahead of the vehicle itself, it 
outputs an alarm (alarm sound) to the driver by means of speaker 105, and automatic braking 
device 106 is controlled to automatically brake or stop the vehicle itself. When the object 
present ahead of the vehicle itself is detected, first of all, the movement velocity in the lateral 
direction (horizontal direction) of the object present in image 3a is detected, and a velocity 
image is formed, with the movement direction and movement velocity in the lateral direction 
of the object in image 3a) displayed in graduated values. 

[0037] That is, the image taken with camera 101 is read from image memory 102, and 

the edges of the object present in image 3a are extracted by means of transformation to binary 
values with a prescribed threshold. For each extracted edge in the image, finer line processing 
is performed to determine the center of the edge correctly. The edge in fine line form is then 
expanded so that the edge has a prescribed width, such as that corresponding to 3 pixels. By 



normalizing the extracted edge, it is possible to obtain an edge image with a uniform width of 
the edges. 

[0038] Figures 4A -4C are diagrams illustrating an example of the processing 

performed to normalize the extracted edges to obtain an edge image. As shown in Figure 4A, 
the obtained edge in binary form is subjected to finer line processing to obtain an edge in a 
finer line form, as shown in Figure 4B. The edge in finer line form is then expanded to have a 
constant edge width as shown in Figure 4C. 

[0039] The count value of the pixel counter corresponding to the pixels with an edge 

present in them in the current edge image is then refreshed in the pixel counter stored in 
counter memory 104. The pixel counter is a counter corresponding to the pixels of the edge 
image; the count value of the pixel counter corresponding to a pixel where an edge is present 
is incremented by 1, and the count value of the pixel counter corresponding to a pixel where 
the edge is absent is initialized to 0. The refreshing processing of the count value is carried 
out for each of the fi*ames taken consecutively with camera 101 . As a result, the pixels with a 
longer edge presence time have a larger count value, and the pixels with a shorter edge 
presence time have a smaller count value. 

[0040] In the edge image, the difference in count values of the pixel counter for pixels 

adjacent to each other in the lateral direction is taken so that the difference in edge presence 
time is computed at each pixel, and the time needed to move the edge by 1 pixel is obtained. 
Then, obtaining the reciprocal of this value enables computing the velocity in the lateral 
direction in the image space for each pixel. The velocity in the lateral direction in the image 
space at each pixel corresponds to the movement velocity in the lateral direction of the edge 
contained in each pixel. As a result, it is possible to compute the velocity component of the 
edge in each pixel of the image, that is, the movement direction and movement velocity of the 
edge. 

[0041] A velocity image is formed, with the velocity component of the object present 

in image 3a) computed in the processing represented by the prescribed grade value. As shown 
in velocity image 5a in Figure 5, as the grade value of the velocity component in the velocity 
image in this embodiment, the pixels for which the velocity has been detected are represented 
by round dots, with higher movement velocities being represented by larger dots. Also, the 
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velocity towards the right side is represented by solid dots, while the velocity towards the left 
side is represented by circles. That is, as shown in Figure 5, a velocity from road edge B on 
the right hand side of the road to the left hand side of the image is detected, and the velocity 
to the right hand side of the image is detected from road edge A on the left hand side of the 
road and from pedestrian C. Also, it is possible to judge from the size of the dots that the 
movement velocity detected for pedestrian C is higher than that detected for road edges A 
and B. 

[0042] A region for detecting the boundary line is then established for delecting the 

road edge or other boundary line on the road on computed velocity image 5a. When the 
images of the scene from ahead of the vehicle are taken while the vehicle is running, the 
position where the road edge is present as the boundary line on the road on velocity image 5a 
is predicted, and a region with a prescribed size is set at that position. For example, as shown 
in Figure 6, regions 6a-6h for detecting the boundary line are symmetrically arranged 
left/right so that the road edges present in velocity image 5a can be detected. A judgment is 
then made as to whether oblique lines having a velocity component exist in the regions 6a-6h 
for detecting the boundary line. That is, a judgment is then made as to whether pixels having 
a velocity component are obliquely arranged side-by-side in the regions 6a-6h for detecting 
the boundary line. 

[0043] Then, for example, assuming that oblique lines detected in regions 6d and 6h 

for detecting the boundary line are detected as a left/right pair, if the detected oblique lines 
have velocity components in different directions, it is judged that the oblique lines detected in 
the regions 6a-6h for detecting the boundary line are road edges. That is, when the oblique 
lines detected in the regions 6d and 6h for detecting the boundary line have velocity 
components in different directions, it is judged that the oblique lines in regions 6d and 6h for 
detecting the boundary line are road edges. 

[0044] The start point and the end point, etc., are extracted from the oblique lines, 

detected as road edges in the regions for detecting the boundary line, as the representative 
points. For example, representative point 6i is extracted from region 6a for detecting the 
boundary line, and left representative point 6j is extracted from region 6e) for detecting the 
boundary line. Double regression analysis is carried out on the basis of the extracted 
representative points, and the equation of the oblique lines in the XY coordinate system is 
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derived. As a result, the two-dimensional road model represented by Equation (1), that is, the 
imaged road model, is obtained. 

x^al-y'^+bl-y + cl ' -(I) 

[0045] As shown in Figure 7, center 7a of the two-dimensional road model 

corresponding to the predicted movement path of the vehicle is computed. 

[0046] Then, as will be explained later, pixels having a velocity component toward 

center 7a of the two-dimensional road model are extracted. Among the extracted pixels, 
pixels having the same velocity component are grouped and are detected as an individual 
object. First of all, as shown in Figure 7, a region in the velocity image covering a prescribed 
range, such as a region of 1 pixel in the longitudinal direction and 3 pixels in the lateral 
direction, is established as small region 7b for detecting the lower end, and the small region 
7b for detecting the lower end is scanned in the velocity image from the lower side upward in 
the Y-axis direction. The points where two or more pixels in small region 7b for detecting the 
lower end are present (distributed) with the same velocity component towards center 7a of the 
two-dimensional road model are then detected. At that time, pixels positioned at the lower 
end of the small region 7b for detecting the lower end are detected as object lower end 
position 7c. 

[0047] The histogram of the velocity image is then computed from object lower end 

position 7c in the Y-axis direction. That is, the histogram that represents the distribution of 
number of degrees in the X-axis direction of the pixels having the same velocity component 
towards center 7a of the two-dimensional road model and present from object lower end 
position 7c in the Y-axis direction is computed and assuming the coordinate of the object 
lower end position 7c is yd and the degree number of the pixels having the same velocity 
component towards center 7a in the two-dimensional road model in the Y coordinate value is 
V(yi), the sum of the degree numbers in the range of the preset standard height ym of 
pedestrian C is computed using following Equation (2). 

Xy{y,)>Tl . . - (2) 

10048] When the sum of the degree numbers computed using Equation (2) exceeds a 

prescribed value Tl, it is judged that there exists an object moving towards center 7a of the 
two-dimensional road model in the range where the pixels having a velocity component 
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towards center 7a of the two-dimensional road model are distributed. When it is judged that 
there exists a moving object at the position higher than preset standard height of 
pedestrian C, the position right before degree number V^i) having the velocity component 
towards center 7a of the two-dimensional road model in the y-coordinate value is less than 
prescribed value T2, such as the position indicated by portion 8a in Figure 8, is detected as 
the upper end position where the moving object is present, that is, object upper end position 
8a. 

[00491 In addition, for the region between object lower end position 7c and object 

upper end position 8a, a straight line is drawn parallel to Y-axis and passing through the 
outermost pixel of the pixels having the same velocity component towards center 7a of the 
two-dimensional road model and present continuously in the X-axis direction. The region 
defined by the straight line parallel to the Y-axis as well as object lower end position 7c and 
object upper end position 8a is defined as region 8b where the moving object is present 
(object presence range). As a result, the pixels in velocity image 5a having the same velocity 
component towards center 7a of the two-dimensional road model are grouped, and it is 
possible to detect object presence range 8b including pedestrian C that moves towards center 
7a from the side in the two-dimensional road model. 

[0050] In velocity image 5a detected in the processing, the positions of road edges A, 

B and object presence range 8b are transformed to positions in real space. In this 
embodiment, the pitching generated by running of the vehicle itself is taken into 
consideration, and it is transformed into the position in real space, as will be explained later. 

[0051] For example, during the movement downward, that is, in the sinking direction 

due to pitching of the vehicle itself, the line of sight orientation of camera 101 is downward 
with respect to that with horizontal road surface as shown in Figure 2. Consequently, in this 
case, as shown in Figure 9, movement takes place in velocity image 5a upward from the 
position of road edges A and B at time t to the position of road edges A and B at time t + L 
Consequently, for the oblique line showing road edge A, the velocity component of 
movement to the right side in velocity image 5a is detected, and, for road edge B, the velocity 
component of movement to the left side in velocity image 5a is detected. 
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[0052] On the other hand, when the vehicle itself moves upward, that is, in the 

floating direction due to pitching, the line of sight orientation of camera 101 is upward from 
that when the road surface is horizontal as shown in Figure 2, and in conjunction with this 
pitching, the position of road edges A and B on velocity image 5a moves downward. As a 
result, for the oblique line that indicates road edge A, the velocity component of movement to 
the left side in velocity image 5a is detected, and, for the oblique line indicating road edge B, 
the velocity component of movement to the right side in velocity image 5a is detected. 

[0053] Taking into consideration the feature, a judgment is made that the point of 

change in the velocity direction of the oblique lines indicating road edges A and B in the 
image of the oblique lines is the pitching balance point, that is, the line of sight orientation of 
camera 101 is horizontal with respect to the road surface. Since the line of sight orientation of 
camera 101 is parallel to the road surface in velocity image 5a at the pitching balance point, 
as explained above, it is possible to compute the three-dimensional coordinates of the 
representative points by means of triangulation based on the coordinates of the representative 
points arbitrarily set on road edges A and B, and on the camera parameters of camera 101. 

[0054] Consequently, as shown in Figure 10, any representative points 1 Oa and 1 Ob 

are set on road edges A and B in velocity image 5a, and based on the coordinates of the 
representative points and the camera parameters of camera 101, the three-dimensional 
coordinates of the representative points can be computed by means of triangulation. Based on 
the three-dimensional coordinates of the representative points, the positions of the oblique 
lines indicating road edges A and B are transformed to positions in real space, and the 
three-dimensional road model, that is, the real space road model, is computed. As a result, it 
is possible to transform the positions of road edges A and B represented by the 
two-dimensional road model to positions in a three-dimensional road model. 

[0055] Also, the position of pedestrian C contained in object presence range 8b in real 

space is computed based on the coordinates of object central point 10c established in object 
presence range 8b in velocity image 5a, and on the camera parameters of camera 101 . As a 
result, it is possible to transform the position of pedestrian C in velocity image 5a to the 
position in real space. Also, based on the relative positional relationships among road edges 
A and B and pedestrian C in real space, the distance from the vehicle itself to pedestrian C 
and the distance from pedestrian C to the road edge are computed. Also, the distance from the 



-9- 



vehicle itself to pedestrian C is not the linear distance from the vehicle itself to pedestrian C. 
Instead, it is the distance between the position on the vehicle line corresponding to the 
position of pedestrian C and the position of the vehicle itself. 

[0056] On the other hand, because the line of sight orientation of camera 101 is not 

parallel to the road surface, in velocity image 5a not at the pitching balance point as 
explained above, the positions of road edges A and B and pedestrian C transformed to 
velocity image 5a at the pitching balance point and the image pickup time interval between 
velocity image 5a at the pitching balance point, and the current velocity image 5a, enable the 
positions in real space to be estimated. Then, based on the relative positional relationships 
among the estimated positions of road edges A and B and pedestrian C in real space, the 
distance between the vehicle itself and pedestrian C and the distance between pedestrian C 
and the road edge are estimated. 

f0057] Based on the relative positional relationships among road edges A and B and 

pedestrian C, the degree of collision danger between the vehicle itself and pedestrian C is 
judged using the following three levels 1-3. 

[0058] (1 ) Level 3. When it is judged that the degree of collision danger between the 

vehicle itself and pedestrian C is the highest from the relative position relationships between 
the road edges and pedestrian C, the degree of collision danger is taken to be level 3. That is, 
as shown in Figure 1 1, when pedestrian C moves inside the road edge present on the left/right 
sides indicated by portion 1 la, that is, when the pedestrian moves inside the vehicle lines 
where vehicle 1 Id is itself running, the degree of collision danger between vehicle 1 1 d itself 
and pedestrian C is judged to be at level 3. 

[0059] (2) Level 2. When it is judged that the degree of collision danger between the 

vehicle itself and pedestrian C is not as high as level 3 yet there is still the danger of collision 
from the relative position relationships between the road edges and pedestrian C, the degree 
of collision danger is taken to be level 2. In other words, when pedestrian C moves in the 
range indicated by portion lib, that is, when the pedestrian moves from outside the road edge 
in a range where the distance from the road edge to pedestrian C is xO (m) or less, the degree 
of collision danger between vehicle lid itself and pedestrian C is judged to be at level 2. The 
distance of xO (m) is computed from the movement velocity of pedestrian C detected in the 
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processing, the distance from vehicle lid itself and pedestrian C, and the vehicle speed 
detected with vehicle speed sensor 107. 

[0060] (3) Level 1 . When it is judged that there is no collision danger between the 

vehicle itself and pedestrian C, the degree of collision danger for pedestrian C is taken to be 
level I . In other words, when pedestrian C moves in the range indicated by portion 1 Ic, that 
is, when the pedestrian moves in from beyond the range of xO (m) from the road edge, the 
degree of collision danger between vehicle 1 Id itself and pedestrian C is judged to be at level 
I . When the judgment result of the degree of collision danger is level 1, there is no need to 
perform vehicle control to avoid collision, to be explained later, because there is no danger of 
collision between the vehicle itself and pedestrian C. 

[0061] Because the degree of collision danger changes depending on the distance 

from the vehicle itself to pedestrian C, the distance from the vehicle itself to pedestrian C is 
classified according to the distance computed using the following Equations (3)-(5). Based on 
the classification result, as will be explained later, vehicle control is performed, as will be 
explained later, to avoid collision of the vehicle itself with pedestrian C. In following 
Equations (3)-(5), V represents the speed of the vehicle 1 Id itself, g represents the 
gravitational acceleration, zO represents the distance 1 le from the position on the vehicle line 
corresponding to the position of pedestrian C and the target stop line. 

z 3 = ( 1 / 2 ) * ( V2 * 0 . 5 g ) + V * 2 + 2 0 •-•(3) 

22=(1/2)*(V2-o.5g)+V*1+20 (4) 
2l=(1/2)*(v2*o.6g)+zO (5) 

[0062] Based on the degree of collision danger assessed in the processing and the 

distance between the vehicle itself and pedestrian C classified according to Equations (3)-(5), 
vehicle control is performed to avoid collision between the vehicle itself and pedestrian C 
according to the following listed three control modes (A)-(C). 

[0063] (A) Caution. When the degree of collision danger is level 3, and the distance 

between the vehicle itself and pedestrian C is longer than z2 and shorter than z3, the control 
mode is "Caution." When the control mode is "Caution," a caution alarm is output from 
speaker 105 to alert the driver to the presence of a moving object ahead of the vehicle with 
which there is a danger of collision. 



[0064] (B) Warning. When the degree of collision danger is level 2, or when the 

degree of collision danger is level 3 and the distance between the vehicle itself and pedestrian 
C is shorter than z2, the control mode is "Warning." When the control mode is "Warning," a 
warning alarm is output from speaker 1 05 to alert the driver to the presence of a moving 
object ahead of the vehicle with which there is a high danger of collision. Also, this warning 
alann is different from that for "Caution" in the (A), For example, the volume for the warning 
alarm may be higher than that for the caution alarm, or, the caution alarm and warning alarm 
may have different rhythms. 

[0065] (C) Automatic braking. When the degree of collision danger is level 3, and the 

distance between the vehicle itself and pedestrian C is shorter than zl , the control mode is 
"Automatic braking." When the control mode is "Automatic braking," automatic braking 
device 106 is controlled to forcibly brake the vehicle itself. That is, when there is a high 
danger of a collision with pedestrian C, there may be no time for the driver to perform the 
braking operation after hearing the caution or warning alarm. Consequently, automatic 
braking device 106 is in this case forcibly controlled to brake the vehicle itself to stop it. 
Also, a "warning" is issued while the control mode is "Automatic braking." 

[0066] Figures 12A and 12B are a flow chart illustrating the processing performed by 

object detecting device 100 in the first embodiment. In the processing shown in Figure 12A, 
the ignition switch of the vehicle is turned on. The power of object detecting device 1 00 is 
turned on, and the operation is executed by the program started by controller 103. In step SI, 
image 3a captured with camera 101 from in front of the vehicle is fetched from image 
memory 102. Then, process flow continues to step S2, and edge extraction processing is 
performed on image 3a to extract the contour of the object present in image 3a to generate an 
edge image. Process flow then proceeds to step S3. As explained above, in step S3 the edge 
velocity information for each pixel is computed, and the computed velocity information is 
transformed into a prescribed grade value to get the velocity image. Then, process flow 
continues to step S4. 

[0067] In step S4, the region for detecting the boundary line in the computed velocity 

image is established, and process flow continues to step S5. In step S5, the oblique line is 
detected by judging whether the pixels having the velocity component in the region for 
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detecting the boundary line are arranged side by side to form an oblique line. Then, process 
flow continues to step S6, and it is determined whether the oblique lines detected in the 
region for detecting the boundary line and forming a left/right pair have velocity components 
in the different directions that indicate they are the road edges. If it is determined that the 
detected oblique lines are not road edges, process flow continues to step S22, to be explained 
below. On the other hand, if it is determined that the detected oblique lines are the road 
edges, process flow continues to step S7. 

[0068] In step S7, as explained above, the two-dimensional road model, that is, the 

imaged road model, is computed, and process flow continues to step S8. In step S8, the pixels 
having a velocity component towards the center of the two-dimensional road model are 
extracted. Then, process flow continues to step S9. Among the extracted pixels, the pixels 
having the same velocity component are grouped and the group is detected as an individual 
object. Process flow then proceeds to step SIO. In step SIO, grouping of the extracted pixels 
is completed, and a judgment is made as to whether all of the objects present in the velocity 
image have been detected. When it is judged that all of the objects present in the velocity 
image have been detected, process flow continues to step S 1 1 . 

[0069] In step SI 1 , a change in the velocity direction in the image of the oblique lines 

indicating the road edges is detected, and a judgment is made as to whether this is the 
pitching balance point. When it is judged to be the pitching balance point, process flow 
continues to step S12. In step S12, the positions of the oblique lines indicating the road edges 
are transformed to positions in real space, and the three-dimensional road model is computed. 
Then, process flow continues to step SI 4, and the relative positional relationship between the 
road edge and the detected object in real space, that is the distance between the road edge and 
the detected object and the distance between the vehicle itself and the detected object, are 
computed. Process flow then proceeds to step SI 6, as will be explained later. 

[0070] On the other hand, when it is judged not to be the pitching balance point, 

process flow continues to step SI 3. In step SI 3, as explained above, based on the image 
pickup time interval between the velocity image at the pitching balance point and the current 
velocity image, the three-dimensional road model is estimated, and process flow continues to 
step SI 5. In step SI 5, the relative positional relationship between the road edge and the 
detected object, that is, the distance between the road edge and the detected object and the 
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distance between the vehicle itself and the detected object, is estimated, and process flow 
continues to step SI 6. 

(00711 In step SI 6, based on the relative positional relationship between the road 

edge and the detected object, the degree of collision danger between the vehicle itself and the 
detected object is judged as one of the levels 1-3. If the degree of collision danger is judged 
to be level 1, no vehicle control to avoid collision is performed since there is no danger of 
collision between the vehicle itself and the detected object, and process flow continues to step 
S22, to be explained later. On the other hand, when it is judged that the degree of collision 
danger is level 2 or 3, process flow continues to step SI 7. In step SI 7, based on the distance 
between the vehicle itself and the detected object, the mode of vehicle control for avoiding 
collision is assessed. 

[0072] When the control mode is the "Caution" mode, process flow continues to step 

SI 9. In this step, the caution alarm is output from speaker 105, and process flow then 
proceeds to step S22, to be explained later. On the other hand, when the control mode is 
"Warning" or "Automatic brake," process flow continues to step SI 8. In this step, a warning 
alarm is output from speaker 105, and process flow continues to step S20. In step S20, a 
judgment is made as to whether the control mode is "Automatic brake." If the control mode is 
not "Automatic brake," process flow continues to step S22, to be explained later. On the other 
hand, if the control mode is "Automatic brake," process flow continues to step S2 1 . 

[0073] In step S21, automatic braking device 106 is controlled to forcibly brake the 

vehicle itself Then, process flow continues to step S22. In step S22, a judgment is made as to 
whether the ignition switch of the vehicle itself is OFF. If it is not OFF, flow returns to step 
SI, and the processing is repeated. On the other hand, if it is judged that the ignition switch 
of the vehicle itself is OFF, the processing comes to an end. 

[00741 The following features and advantages can be realized in the first 

embodiment: 

[0075] (1) The region for detecting the boundary lines is established in the computed 

velocity image on the basis of the image captured by camera 101 . If the oblique lines detected 
in the region for detecting the boundary lines form a pair and have velocity components in 
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different directions, it is determined that the left/right pair of oblique lines detected in the 
region for detecting the boundary lines are the boundary lines on the road, that is, the road 
edges. As a result, the road edges contained in the image captured by camera 101 have a 
linear component parallel to the optical axis, so that they can definitely be picked up as 
oblique lines on the image. In addition, taking into consideration the fact that a moving object 
can easily appear within the oblique lines, it is possible to detect the road edges with high 
accuracy. 

[0076] (2) The point at which the velocity direction in the image of the oblique lines 

indicating road edges A and B changes is designated as the pitch balance point. As a result, 
by simply measuring the velocity direction of the oblique lines in the image, it is possible to 
detect the pitch balance point. 

[0077] (3) On the basis of the coordinates of the representative points at road edges A 

and B at the pitch balance point and the camera parameters of camera 101, the 
three-dimensional coordinates of the representative points can be computed by means of 
triangulation. Also, at the pitch balance point, the camera parameters are in approximate 
agreement with those in the initial state of mounting, and the precision with which distance 
can be measured using a triangulation scheme is high. Taking this fact into consideration, it is 
possible to compute three-dimensional coordinates with great accuracy. 

[0078] (4) On the basis of the relative positional relationship between road edges A 

and B and pedestrian C in real space, the degree of collision danger between the vehicle and 
pedestrian C can be assessed, and when the distance between the vehicle and pedestrian C is 
also taken into consideration, it is possible to control the vehicle to avoid a collision with 
pedestrian C. As a result, it is possible to make a correct judgment of the degree of danger, 
and it is possible to avoid a collision with pedestrian C. 

Second Embodiment 

[0079] In the second embodiment, not only the road edges present on both sides of 

the vehicle and detected in the first embodiment but also various types of boundary lines on 
the road, such as the boundary lines of muhiple lane dividing lines on a multi-lane road or the 
boundary lines of road and buildings, are detected. 
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[00801 The block diagram shown in Figure 1, the diagram shown in Figure 2 for 

illustrating an arrangement example of camera 101 on a vehicle, and the diagram shown in 
Figure 4 for illustrating an example of edge normalization are the same as in the first 
embodiment and will not be explained again. Calculation of the 2-dimenisonal road model on 
the basis of the detected road edges, detection of the object field including the object facing 
the center of the 2-dimensional road model, the judgment of the degree of risk on the basis of 
the positional relationship between the object and the boundary lines on the road transformed 
on the 3-dimensional road model, and the vehicle control or warning operation on the basis of 
the result of the degree of risk judgment explained on the basis of Figures 7-1 1 are also 
carried out in the same way as described in the first embodiment and will not be explained 
again. 

[0081] Figure 13 shows a detailed example of an image picked by camera 101 in the 

second embodiment. As in the first embodiment, image 13a is taken with the origin in the 
upper-left corner of the image and is represented with an xy coordinate system (camera 
coordinates) with the x-axis defined from left to right and the y-axis defined from top to 
bottom. In the example shown in Figure 13, image 13a includes road edges D and E formed 
as lane markers, white lines or guardrails or the like positioned on the left/right sides of the 
road, boundary line F with the adjacent lane dividing line, and pedestrian G present in front 
of the vehicle. 

[0082] Velocity image 14a shown in Figure 14 is formed by carrying out the same 

image processing as described in the first embodiment on image 13a shown in Figure 13. 
Multiple regions for detecting boundary lines are formed by dividing a prescribed field in the 
velocity image 14a. In other words, as shown in Figure 15, a prescribed field in front the 
vehicle, for example, a distance of 20-60 m from the vehicle in real space is divided into 6 
parts having prescribed height in the vertical direction (y-axis direction) of the image. Each 
region divided in the vertical direction is divided at equal intervals in the horizontal direction 
of the image to form multiple regions 15a for detecting boundary lines in a field on the image 
corresponding to the prescribed field in front of the vehicle. 

[0083] Each region 1 5a for detecting boundary lines formed as described above is 

further divided into small regions having a prescribed size. In this embodiment, for example, 
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each region 15a for detecting boundary lines is equally divided into 64 small regions. In this 
way, each region 15a for detecting boundary lines is comprised of 8x8 small regions. The 
candidates for the boundary lines on the road present in velocity image 14a in each divided 
small region are detected. 

[0084] As in the first embodiment, the road edges or other boundary lines on the road 

are detected as oblique lines in velocity image 14a. Taking this fact into account, the 
candidates for the oblique lines in each small region, that is, the candidates for the boundary 
lines on the road can be detected by detecting the pixels having parallel velocity components 
in the oblique direction to form the oblique lines in each small region. 

[0085] First, any pixel present in each small region is taken as a target pixel. It is 

determined whether the target pixel constitutes an oblique line with the adjacent pixels in the 
oblique direction. In order to determine whether the target pixel and the adjacent pixels 
constituting a straight line obliquely have respective velocity components, for example, the 
spatial filter for detecting candidates for oblique lines shown in Figure 16B is used on the 
target pixel. In this case, the spatial filter is applied such that the center 16a of the spatial 
filter matches the target pixel. 

[0086] In velocity image 14a, the image of a boundary line on the road present on the 

right side of the vehicle is usually taken as an oblique line with a higher left end as shown by 
road edge E or boundary line F with the adjacent lane dividing line, and the image of a 
boundary line on the road present on the left side of the vehicle is usually taken as an oblique 
line with a higher right end as shown by road edge D. Therefore, a spatial filter used for 
detecting an oblique line with a higher left end, that is, the spatial filter shown in Figure 16B 
is applied to the target pixel present in the right half of velocity image 14a, and a spatial filter 
used for detecting an oblique line with a higher right end, that is, a spatial filter obtained by 
reversing the spatial filter shown in Figure 16B in the left and right direction is applied to a 
target pixel present in the left side of velocity image 14a. 

[0087] If the result of applying a spatial filter to a target pixel shows that the target 

pixel and the pixel above it and/or the pixel below it in the oblique direction have their 
respective velocity components, that target pixel is detected as the candidate for a dot 
forming the oblique line, that is, as a dot candidate. This processing is carried out with 
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respect to all of the pixels in the small region to detect all the dot candidates in the small 
region. If 3 or more dot candidates in a small region are side by side in the oblique direction 
to form an oblique line in that small region and these dot candidates satisfy the slope 
condition to be explained below, the straight line formed by connecting these dot candidates, 
that is, the oblique line, is detected as a candidate for a boundary line on the road (boundary 
line candidate). 

[0088] In the following, the slope condition for detecting a boundary line candidate 

will be explained. In general, if there are multiple boundary lines on the road on the right or 
left side viewed from the vehicle in velocity image 14a, the slope of a boundary line close to 
the vehicle is larger than the slope of a boundary line away from the vehicle. In other words, 
a boundary line close to the vehicle has a larger angle formed with the x axis than a boundary 
line away from the vehicle. For example, in the example shown in Figure 14, the slope of 
road edge E present on the right side of the vehicle is larger than the slope of boundary line F 
with the adjacent lane dividing line. 

[0089] Taking this fact into account, whether a straight line formed by connecting the 

dot candidates detected as described above is a boundary line candidate can be determined as 
follows. For example, as shown in Figure I6A, when a straight line formed by connecting dot 
candidates is detected from small region 16b, it is determined whether straight lines formed 
by connecting dot candidates are detected from other small regions on the same side viewed 
from the vehicle and at the same height. If no straight line formed by connecting dot 
candidates is detected from other small regions on the same side viewed from the vehicle and 
at the same height as small region 16b, the straight line formed by connecting the dot 
candidates in small region 16b is detected as a boundary line candidate. 

[0090] On the other hand, if a straight line formed by connecting dot candidates is 

detected from another region on the same side viewed from the vehicle and at the same height 
as small region 16b, such as, small region 16c, the slopes of the straight lines formed by 
connecting the dot candidates detected in these small regions are compared to determine 
whether the slope of the straight line close to the vehicle is larger than the slope of the 
straight line away from the vehicle. If the slope of the straight line close to the vehicle is 
larger than the slope of the straight line away from the vehicle, both are detected as boundary 
line candidates that constitute part of the boundary line on the road. On the other hand, if the 
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slope of the straight line close to the vehicle is smaller than the slope of the straight line away 
from the vehicle, both are excluded as boundary line candidates. 

[0091] The continuity of each of the detected boundary line candidates is judged as an 

oblia ue line in the vertical direction in velocity image 14a. If continuity is confirmed for 
multiple boundary line candidates, the straight line formed by connecting these boundary line 
candidates is detected as a boundary line on the road. In this way, as shown in Figure 17, 
boundary lines D, E, and F on the road can be detected from velocity image 14a. 

[0092] Figures 1 8A and 1 8B are a flow chart illustrating the processing of object 

detecting device 100 in the second embodiment. The processing shown in Figure 18A is 
carried out as a program started by control device 103 when the ignition switch of the vehicle 
is turned on and the power object for detecting device 100 is turned on. In Figure ISA, the 
same step numbers are assigned to the same processes as those that are described in the first 
embodiment shown in Figure 12, and the explanation focuses on the differences of these 
embodiments. 

[0093] In steps S4-2, multiple regions 15a for detecting a boundary line are set on 

velocity image 14a as described above, each region 15a for detecting a boundary line is 
further divided into small regions with a prescribed size. Then, process control goes to steps 
S5-2. The spatial filter shown in Figure 16B is applied to the target pixel in each small 
region to detect the dot candidates. If 3 or more detected candidates are side by side in the 
oblique direction to form an oblique line in the small region and the straight line formed by 
connecting these dot candidates satisfies the slope condition, the straight line formed by 
connecting these dot candidates is detected as a boundary line candidate. 

[0094] Process control then goes to step S6-2. The continuity of each of the boundary 

line candidates detected on velocity image 14a is judged as an oblique line in the vertical 
direction in velocity image 14a to determine whether the straight line formed by connecting 
these boundary line candidates is detected as a boundary line on the road. If a boundary line 
on the road is detected, process control goes to step S7. If no boundary line is detected, 
process control goes to step S22. 
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[00951 According to the second embodiment explained above, multiple regions for 

detecting a boundary line are formed in a prescribed field of velocity image 14a, for example, 
a distance of 20-60 m from the vehicle in real space. Candidates for the oblique lines are 
detected in each of the regions for detection of boundary lines, and the boundary lines on the 
road are detected. Consequently, in addition to the effects described in the first embodiment, 
in addition to the road edges present on both sides of the vehicle, various types of boundary 
lines on the road, such as the boundary lines of multiple lane dividing lines on a multi-lane 
road or the boundary lines of road and buildings, can also be detected. 



[0096] Referring to Figures 19 and 20, an implementation of the processing 

performed to normalize the extracted edges to obtain an edge image, as shown in Figure 4. 

[0097] The edge extracting section 202 extracts an edge of an image using a SOBEL 

filter for the frame image inputted from the camera 101 by generating an edge image. The 
edge standardizing section 203 standardizes an edge width of the edge, extracted by the edge 
extracting section 202, to a given pixel number in a shift direction of the object. 

[0098] Now, standardizing operation of the edge standardizing section 203 for the 

edge width is described in connection with Figures 19 and 20. As shown in Figure 19, the 
edge-width standardizing section 203 is comprised of a binary valuing section 231 that allows 
the edge image, generated by the edge extracting section 202, to be valued in a binary state, a 
thin-line condensing section 232 that converts the edge width of the edge that is valued in the 
binary state by the binary valuing section 231 in a thin line to a given pixel number, and an 
expanding section 233 that expands the edge width of the edge, which is formed in the thin 
line by the thin-line condensing section 232, to a given pixel number. 

[0099] With the edge standardizing section 203 configured in such a structure, if the 

edge image is inputted from the edge extracting section 202, the binary valuing section 231 
executes binary valuing operation on the edge image. During this binary valuing operation, a 
pixel of a pixel area, whose edge is detected, is allocated to take "1" while allocating a pixel 
of a pixel area, whose edge is not detected to be "0" upon which a binary valued image is 
generated as shown in a frame image (a) in Figure 20. 
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[00100] Next, the thin-line condensing section 232 executes thin-line condensing 

operation for the binary valued image that is valued by the binary valuing section 23 1 in a 
binary state. The thin-line condensing operation is operation in which the edge width of the 
detected edge is contracted to a given pixel number. With the frame image (b) shown in 
Figure 20, the edge width of the edge is condensed to a given pixel number of one pixel. 
Upon condensing the line of the edge to the given pixel width in such a way, the thin-line 
condensing section 232 sets a center position to be a center of the edge. 

[00101] Subsequently, the expanding section 233 executes expanding operation for 

expanding the edge width of the edge whose line is condensed by the thin-line condensing 
section 232. During this expanding operation, the edge width is expanded in both shift 
directions of the edge, that is, in one shift direction away from a center position of the edge, 
set by the thin-line condensing operation, and in the other direction opposite to the one shift 
direction. With the frame image (c) in Figure 20, the edge width is expanded in both 
directions by one pixel, i.e., in the shift direction (positively on the x-axis) away fi-om the 
pixel position X, forming the center of the edge, and the other direction (negatively on the x- 
axis) opposite to the shift direction for thereby standardizing the edge width of the edge in 
three pixels with respect to the shift direction. 

[00102] Upon executing the thin-line condensing and expanding operations in such a 

way, the edge-width standardizing section 203 standardizes the edge width of the edge, 
extracted by the edge extracting section 202, in a given pixel number toward the shift 
directions of the edge. 

[00103] Here, although upon executing the thin-line condensing and expanding 

operations, the edge width is standardized, an alternative may be such that the edge extracting 
section 202 detects a peak position of the detected edge after which the edge width is 
standardized in a way to cause the edge width to have pixel widths each by a given pixel 
number in the shift direction of the edge and in the other direction opposite the shift direction 
with respect to the edge peak position. 

[00104] Next, the balloting section 204 executes the operation to count up the 
standardized edges standardized in a way set forth above. This count-up operation is 
implemented such that values of memory addresses of memory regions whose standardized 
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edges are detected are counted up while initializing the values of memory addresses of pixel 
areas whose standardized edges are not detected. 

[00105] Now, the count-up operation of the balloting section 204 is described with 

reference to Figure 2 1 . 

[00106] As shown in Figure 21, it is supposed that in a frame image (a) at time t, the 

edge width is expanded from a pixel position X-1 to the other pixel direction X+1 in both 
directions, including a shift direction of the edge and the other direction opposite to the shift 
direction, by one pixel, respectively. 

[00107] In this case, the balloting section 204 counts up ballot values of the pixel 

positions X~l , X and XH-l , whose standardized edges are detected by the count-up mask 141 
one by one, and a ballot value of the pixel areas, whose standardized edges are not detected, 
is reset. 

[00108] In a frame image (a) in Figure 21, since the standardized edges are detected 

on the pixel positions X-1, X and X+1 at time t, the ballot values are counted up at the 
respective pixel positions one by one such that the pixel position X+1 takes the ballot value 
of 1, the pixel position X takes the ballot value of 3 and the pixel position X-1 takes the 
ballot value of 5. 

[00109] No edge shifts at time t+1, the standardized edges are detected at the pixel 

positions X-1, X and X+1 and, as shown in the frame image (b) in Figure 21, the ballot 
values of the pixel positions X-1, X and X+1 are further counted up one by one such that the 
pixel position X-1 takes the ballot value of 2, the pixel position X takes the ballot value of 4 
and the pixel position X+1 takes the ballot value of 6. 

[00110] Further, at time t+2, the edge shifts in a positive direction on the x-axis by 

one pixel upon which the standardized edges are detected at the pixel positions X, X-M and 
X+2. Accordingly, the ballot values of the pixel positions X, X+1 and X+2, whose 
standardized edges are detected, are counted up, while resetting ballot value of the pixel area 
X-1 whose standardized edge is not detected. As a result, as shown by a frame image (c) in 
Figure 21, the pixel position X+2 takes the ballot value of 1, the pixel position X+1 takes the 
ballot value of 3 and the pixel position X takes the ballot value of 5, In addition, the ballot 
value of the pixel position X-1, whose standardized edge is not detected, is reset to "0". 
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[00111] In such a way, the balloting section 204 counts up the ballot values of the 

pixel positions whose standardized edges are detected, while resetting ballot values of the 
pixel areas whose standardized edges are not detected. 

[00112] While in Figure 21, the ballot values are detected at a sum of three positions, 

i.e., the pixel positions X-1, X and X+l, as the pixel areas of the standardized edges, the 
ballot values of any positions may be detected provided that the gradient of the ballot values 
is obtained as will de described below. 

[00113] Further, if the frame rate is set to be sufficiently higher than the speed in 

which the edge (at a central pixel position of the standardized edge) shifts, the standardized 
edges are detected a number of times on the same pixel areas for frame images successively 
appearing in time series. In case of Figure 21, the standardized edge in the pixel position X is 
detected two times at times t and t+1 . Consequently, the ballot value, resulting when the 
ballot values of the pixel areas whose standardized edges are detected, is substantially 
equivalent to a time interval (frame number) during which the standardized edges are 
detected in the relevant pixel area. Particularly, this means how many number of frames are 
needed after the edge has shifted for the minimum ballot value h, among the ballot values of 
the pixel areas on which the standardized edges are located, to be located on the same pixel 
area. 

[001 14] Subsequently, the shift speed detecting section 205 calculates a shift speed, a 

shift direction and a position of the edge. The shift speed detecting section 205 initially 
calculates a gradient of the ballot values of the pixel areas of the standardized edges and 
depending on the gradient of the ballot value, calculates the shift direction, the shift speed 
and the position of the edge. 

[00115] Hereunder, this calculation method is described with reference to Figure 21. 

In case of the frame image (b) in Figure 21, the pixel positions X-1, X and X+l have the 
ballot values of 6, 4 and 2, respectively. Therefore, upon subtracting the ballot value 2 at the 
pixel position X-f 1 from the ballot value 6 at the pixel position X-1, the gradient of the ballot 
value can be calculated as H = (6*2)/2 = 2. This means 

[00116] H = {(Time Interval needed for Standardized Edge to shift from Pixel 

Position X-1 to Pixel Position X+1)} / (2 Pixels) 
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[00117] Accordingly, the gradient H of the ballot values is substantially equivalent to 

the time interval (frame number) needed for the standardized edges to pass across the pixel 
position X. That is, the gradient H of the ballot values is equivalent to obtain information 
about how many number of frames are needed for the standardized edge to shift by one pixel 
and the shift speed 1/H of the edge can be calculated based on the gradient H of the ballot 
value. 

(001 18] In the frame image (b) in Figure 2 1 , four frames are needed for the 

standardized edge to shift by one pixel and, hence, the shift speed of the edge can be 
calculated to be 1/4 (pixel/frame). Likewise, even in the frame image (c) in Figure 2 1, the 
gradient of the ballot value is expressed as H = (5*1)/1 = 4 and, hence, the shift speed of the 
edge is expressed as 1/4 (pixel/frame). 

[001 19] Further, the shift direction of the edge can be judged on a size of the ballot 

value. The pixel area, appearing when the edge shifts and a new standardized edge is 
detected, has a ballot value of 1 that forms a minimal value among the ballot values of the 
respective pixel positions. Consequently, the ballot value in a direction in which the edge 
shifts is small and the ballot value in a direction opposite to the direction in which the edge 
shifts is large, whereby it becomes possible to judge the shift direction of the edge. 

[00120] In addition, if the frame rate is set to be sufficiently higher than the speed at 

which the edge shifts, it is assumed that the object is moving at a constant speed. Moreover, 
among the ballot values of the pixel areas on which the standardized edges are located, the 
minimal ballot value "h" means that the standardized edge is located on the same position for 
a time interval in which the standardized edge is detected at the relevant pixel position, that 
is, the given number of fi-ames needed after the edge has shifted. 

[00121] From the foregoing, suppose a center position of the edge is X, the position 

of the edge can be given as expressed below. 

Current Position of Edge = X + h/H 

[00122] In the frame image (b) in Figure 21 , since at time t+1, the edge is detected in 

the same pixel position two successive frames at the edge speed of 1/4 (pixel/fi-ame), the 
pixel position of the edge at time t+1 can be calculated to assume a position shifted from the 
pixel position by 
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2 (Frame)x{l/4 (Pixel / Frame)} = 0.5 Pixel 



(00123] Upon calculating the shift speed, the shift direction and the position of the 

edge in such a way, the shift speed detecting section 205 transmits the calculated shift speed 
to the edge standardizing section 203. Upon receipt of the shift speed, the edge standardizing 
section 203 alters the edge width of the edge to be standardized. In the frame image (c) in 
Figure 21, although the edge width, subsequent to the standardizing operation, has three 
pixels in respect of the shift direction of the edge, the edge width in respect of the shift 
direction of the edge is altered to be greater than the three pixels when the received shift 
speed is high whereas when the shift speed is low, the edge width in respect of the shift 
direction of the edge is altered to be smaller than the three pixels. 

[00124] Thus, altering the edge width for standardization enables the edge width to be 

standardized such that the standardized edges overlap one another between the frame images 
successive in time series in accordance with the shift speed, making it possible to expand a 
range of the shift speed available for detection. 

[00125] As set forth above, the motion detection apparatus and motion detection 

method of the presently filed embodiment have advantageous effects, in addition to those of 
the first embodiment, wherein the edge-width standardizing section 203 is structured to allow 
the edge width, to be standardized, to be altered depending on the shift speed calculated by 
the shift speed detecting section 205 whereby even in the presence of variation in the shift 
speed of the object, the edge width can be standardized to allow the standardized edges to 
overlap one another between the successive frames for thereby expanding the range for the 
shift speed of the object available for detection. 

[00126] Additional background information with respect to the edge extraction may be 

found in U.S. Patent Application Ser. No. 1 1/146,041 filed June 7, 2005 and entitled 
"Motion Detection Apparatus and Motion Detection Method," which is hereby incorporated 
by reference in its entirety, and in Japanese Patent Applications Laid-open No. P2004-278250 
with a filing data of September 24, 2004 and Laid-open P2004-279394 with a filing data of 
September 27, 2004, both of which are hereby incorporated by reference. 



[00127] The following modifications may be made to object detecting device 1 00 in 

the embodiment. 
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(1) In the first and second embodiments explained above, edge extraction 
processing is performed on the image that has been captured to obtain an edge image. The 
pixel counter is then refreshed for the pixel where an edge is present, and the velocity 
component of the edge at each pixel in the image is computed on the basis of the count value 
of the pixel counter. However, the present invention is not limited to this scheme. For 
example, other conventional schemes can be adopted, such as a gradient scheme or block 
matching scheme to compute the optical flow. 

[00128] (2) In the first and second embodiments, the grade values shown in Figure 5 

are adopted to compute velocity image 5a. However, the present invention is not limited to 
this scheme. For example, one may make adopt other grade values to compute the velocity 
image. 

[00129] (3) In the first and second embodiments, object detecting device 1 00 of the 

present invention is mounted on a vehicle. However, the present invention is not limited to 
this scheme. For example, it can be mounted on other moving bodies. 

[00130] (4) In the second embodiment, multiple regions for detecting boundary lines 

are formed in a prescribed field of velocity image 14a, for example, a distance of 20-60 m 
from the vehicle in real space. However, the present invention is not limited to this scheme. It 
is also possible to form the region for detecting boundary lines from the entire velocity image 
14a or form regions for detecting boundary lines only in fields with a high probability of 
containing boundary lines on the road, such as a prescribed field to the left and right of 
velocity image 14a. 

[00131] Also, the present invention is not limited to the embodiment, and 

modifications can be made as long as the essence of the present invention is observed. 

[00132] In the following, the correspondence between the structural elements of the 

claims and the embodiments will be explained. The camera 101 corresponds to the image 
pickup means; speaker 105 and automatic braking device 106 correspond to the risk 
avoidance means; controller 103 corresponds to the velocity information computing means, 
pixel extracting means, oblique line detecting means, boundary line detecting means, balance 
point judgment means, object detecting means, transformation means, and degree of collision 
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danger judgment means. Also, such correspondences are merely examples, and the 
correspondences can change depending on the constitution of the embodiment. 

[00133] This application is based on Japanese Patent Application No. 2004-346670, 

filed November 1 1 , 2004 in the Japanese Patent Office, the entire contents of which are 
hereby incorporated by reference, and on Japanese Patent Application No. 2005-123494, filed 
April 21, 2005, the entire contents of which are hereby incorporated by reference. 

[00134] Also, the above-mentioned embodiments have been described in order to 

allow easy understanding of the present invention, and do not limit the present invention. On 
the contrary, the invention is intended to cover various modifications and equivalent 
arrangements included within the spirit and scope of the appended claims, which scope is to 
be accorded the broadest interpretation so as to encompass all such modifications and 
equivalent structures as is permitted under the law. 
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